1. Field of the Invention
The present invention generally relates to artificial intelligence systems, and more particularly to an apparatus that simulates a biological neuron for use in artificial intelligence applications.
2. Discussion of the Related Art
As is known, artificial neural networks are made up of multi-dimensional matrices comprising simple computational units generally operating in parallel. These simple computational units are designed to replicate the behavior and interaction of biological neurons, in an effort to mimic the behavior and computational process of the human brain. Such computational process is desirable for achieving things such as extremely fast decisional output, the ability to reason, and the ability to learn.
Many artificial silicon based neural networks have been made. The complexity of these are orders of magnitude less than that of biological nets. State of the art technology simulates neural behavior only in terms of axodendritic connections (discussed further below) and thus is extremely limited. Typically, artificial neural networks try to mimic the human biological network, which allows systems to be trained, as opposed to being programmed. The training techniques create nonlinear statistical models of input data by adjusting the synaptic weight coefficients. These models tend to be more accurate and more sophisticated than the rigid conventional logic of circuit design (or programming techniques) which attempt to model the same behavior.
Artificial neural networks can be implemented in either hardware or in software, depending on the level of performance required. Conventional neural network architectures are often simplistic feed forward or recurrent models, where the timing of events is not important to the processing being done. In biological systems, each synapse has a different size and has size-appropriate capability. Also synapses possess a metabolic rate, whereby if a particular synapse is an often-used-route, it will be more efficient and can communicate more information per unit volume than a synapse which is less frequently used. These qualities are modeled in artificial networks by `synaptic weight coefficients`, which adjust the input signal. Such models, however, generally take into account only the processing of spatial stimuli, and make no provision for temporal processing.
Unfortunately, this further limits the capabilities of the such models, as they fail to achieve the dynamic manner in which information is processed in biological systems. In this regard, the arrangement of neurons in the biological system actually processes very complex signals, not only spatially but temporally as well. Although there has been some research done in artificial systems which theoretically recognize the neuron's temporal qualities, there has been little practical work which utilizes these frequency parameters to control any function within the network.
For example, U.S. Pat. No. 5,615,305 to Nunally discloses an artificial neural network that provides recursive, asynchronous neural processing in a neural processor element, or neuron, which uses a "pulse stream" signaling mechanism that attempts to simulate that of the biological neuron. As acknowledged within the '305 patent, the invention therein advances the state of by providing an asynchronous processor, as opposed to synchronous processors, which impair the ability to achieve a more natural (biological) operation.
However, a shortcoming of the '305 patent, which is common to many prior art devices, relates to the utilization of transistor circuits to realize the processing elements. While the proliferation of semiconductor technology has paved the way for the wide spread use of transistor-based circuits, such circuits have disadvantages when utilized to mimic systems that are inherently analog in nature. This failing is compounded when modeling large scale networks. More particularly, the failing of a transistor model to accurately mimic a biological neuron is compounded when a large number of these transistor circuits are interconnected to simulate a neural network. Thus, the use of transistor-based processing circuits for implementing artificial neural networks creates error-sensitive circuits, wherein an error in one elements may catastrophically compound as it passes through subsequent processing elements.
U.S. Pat. No. 5,537,512 is another example of a largely transistor-based implementation of a neural processing element. The device of the '512 patent utilizes one or more EEPROMS as analog, reprogrammable synapses, which apply weighted inputs to positive and negative term outputs which are combined in a comparator. As noted in the '512 patent, the device therein addresses the need for flexibly configurable neural network elements which can be conveniently and densely packaged while providing wide ranges of programming options, particularly analog programming connections and weights. One of the benefits realized by the device in the '512 patent is the non-volatility of the synaptic weights stored in EEPROMs, as the device need not be reprogrammed before use, which saves time otherwise expended in the overhead of downloading synaptic weighting coefficients.
Notwithstanding the advances realized by the devices of the '305 and '512 patents, and other devices known in the art, the prior art still fails to accurately and adequately model a biological neuron, and therefore artificial neural networks still fail to realize the effectiveness desired. Accordingly, there is still a need to provide a device that more accurately mimics a biological neuron and, therefore, a device that may be interconnected with many similar devices to more accurately model a biological neural network.